Maximally Informative Statistics
نویسنده
چکیده
In this paper we propose a Bayesian, information theoretic approach to dimensionality reduction. The approach is formulated as a variational principle on mutual information, and seamlessly addresses the notions of sufficiency, relevance, and representation. Maximally informative statistics are shown to minimize a Kullback-Leibler distance between posterior distributions. Illustrating the approach, we derive the maximally informative one dimensional statistic for a random sample from the Cauchy distribution.
منابع مشابه
Maximally informative stimuli and tuning curves for sigmoidal rate-coding neurons and populations.
A general method for deriving maximally informative sigmoidal tuning curves for neural systems with small normalized variability is presented. The optimal tuning curve is a nonlinear function of the cumulative distribution function of the stimulus and depends on the mean-variance relationship of the neural system. The derivation is based on a known relationship between Shannon's mutual informat...
متن کاملMaximally Informative Statistics for Localization and Mapping
This paper presents an algorithm for simultaneous localization and mapping for a mobile robot using monocular vision and odometry. The approach uses Variable State Dimension Filtering (VSDF) flamework to combine aspects of extended Kalman filtering (EKF) and nonlinear batch optimization. This paper describes two primary improvements to the VSDF. The first is to use the maximally informative sta...
متن کاملMaximally Informative Observables and Categorical Perception
We formulate the problem of perception in the framework of information theory, and prove that categorical perception is equivalent to the existence of an observable that has the maximum possible information on the target of perception. We call such an observable maximally informative. Regardless whether categorical perception is " real " , maximally informative observables can form the basis of...
متن کاملMaximally Informative Decision Rules In a Two-Person Decision Problem Maximally Informative Decision Rules in a Two-Person Decision Problem∗
This paper studies how much information can be revealed when agents with private information lack commitment to actions in a given mechanism as well as to the mechanism itself. In a two-person decision problem, agents are allowed to hold on to an outcome in one mechanism while they play another mechanism and learn new information. Formally, decision rule is maximally informative if it is (i) po...
متن کامل